feat(stream): Microsoft 365 directory-audit event type and deserializer - #1603
Open
andriikachur wants to merge 17 commits into
Open
feat(stream): Microsoft 365 directory-audit event type and deserializer#1603andriikachur wants to merge 17 commits into
andriikachur wants to merge 17 commits into
Conversation
Additive support for Entra directory audit events entering the logs
pipeline via the generic message-type path:
- IntegratedToolType.MICROSOFT_365 ("microsoft-365")
- MessageType.MICROSOFT_365_AUDIT_EVENT -> CASSANDRA_EVENT_LOG + KAFKA_PINOT
- UnifiedEventType M365_* additions (RoleManagement/failures -> WARNING)
- EventTypeMapper mappings for Graph directoryAudits category values
- Microsoft365AuditEventDeserializer: toolEventId = Graph audit id
(idempotent upserts), eventTimestamp from activityDateTime,
result=failure -> M365_AUDIT_FAILURE, unmapped category -> M365_AUDIT_OTHER
A2.2 enrichment decision: INTEGRATED_TOOLS_EVENTS cannot supply org fields
for agentless events (machine-lookup only populates enriched data when
agentId is present), so a new PRE_ENRICHED DataEnrichmentServiceType +
PreEnrichedDataEnrichmentService passes organizationId/organizationName/
userId through from the deserialized message (populated from the
pre-enriched payload), with TenantIdProvider fallback for tenantId.
DeserializedDebeziumMessage gains organizationId/organizationName/userId
(additive; existing MessageTypes unaffected).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
📝 WalkthroughWalkthroughMicrosoft 365 and Google Workspace directory audit events are added across event contracts, category mappings, Debezium deserialization, pre-enrichment, documentation, and automated tests. ChangesAudit event ingestion
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant DebeziumMessage
participant AuditEventDeserializer
participant EventTypeMapper
participant PreEnrichedDataEnrichmentService
participant EventDestinations
DebeziumMessage->>AuditEventDeserializer: deliver audit payload
AuditEventDeserializer->>EventTypeMapper: resolve category and failure state
EventTypeMapper-->>AuditEventDeserializer: return unified event type
AuditEventDeserializer->>PreEnrichedDataEnrichmentService: provide normalized message
PreEnrichedDataEnrichmentService-->>AuditEventDeserializer: return tenant and organization data
AuditEventDeserializer->>EventDestinations: route Cassandra and Kafka/Pinot event
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches 💡 1📝 Generate docstrings 💡
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 3
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@openframe-stream-service-core/src/main/java/com/openframe/stream/deserializer/Microsoft365AuditEventDeserializer.java`:
- Line 61: Update Microsoft365AuditEventDeserializer.java:61 to read the
directory audit identifier from id, optionally retaining auditId only as a
fallback; update Microsoft365AuditEventDeserializerTest.java:27-35 to use id and
assert the Graph identifier is populated.
- Around line 97-107: Update Microsoft365AuditEventDeserializer.buildDetails to
also preserve the provider-supplied additionalDetails field alongside
initiatedBy and targetResources, retaining its existing value when present and
non-null. Add or update the Microsoft 365 audit fixture and assertion to verify
additionalDetails is included in the persisted details payload.
- Around line 78-82: Update Microsoft365AuditEventDeserializer’s result
classification to treat both failure and timeout values as M365_AUDIT_FAILURE
before category-based mapping. Add a regression test covering a directory audit
event with result timeout and assert it produces M365_AUDIT_FAILURE.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 892e1031-264a-4090-8422-0b4a4ca2a981
📒 Files selected for processing (12)
openframe-data-cassandra/src/main/java/com/openframe/data/cassandra/model/enums/UnifiedEventType.javaopenframe-data-kafka/src/main/java/com/openframe/data/model/enums/DataEnrichmentServiceType.javaopenframe-data-kafka/src/main/java/com/openframe/data/model/enums/IntegratedToolType.javaopenframe-data-kafka/src/main/java/com/openframe/data/model/enums/MessageType.javaopenframe-stream-service-core/src/main/java/com/openframe/stream/deserializer/Microsoft365AuditEventDeserializer.javaopenframe-stream-service-core/src/main/java/com/openframe/stream/mapping/EventTypeMapper.javaopenframe-stream-service-core/src/main/java/com/openframe/stream/mapping/SourceEventTypes.javaopenframe-stream-service-core/src/main/java/com/openframe/stream/model/fleet/debezium/DeserializedDebeziumMessage.javaopenframe-stream-service-core/src/main/java/com/openframe/stream/service/PreEnrichedDataEnrichmentService.javaopenframe-stream-service-core/src/test/java/com/openframe/stream/deserializer/Microsoft365AuditEventDeserializerTest.javaopenframe-stream-service-core/src/test/java/com/openframe/stream/mapping/EventTypeMapperTest.javaopenframe-stream-service-core/src/test/java/com/openframe/stream/service/PreEnrichedDataEnrichmentServiceTest.java
…itionalDetails Review follow-up (PR #1603): - Graph directoryAudits result can be success|failure|timeout — timeout now also maps to M365_AUDIT_FAILURE instead of falling through to the category mapping as an INFO event - details JSON now carries provider-supplied additionalDetails alongside initiatedBy and targetResources (payload contract extended in the poller and Graph model accordingly) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ails Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Add GoogleWorkspaceAuditEventDeserializer mirroring the Microsoft 365 Entra audit deserializer: events are hand-built by the poller (not CDC), pre-enriched with tenant/organization fields, and carry no agent reference. KafkaMessageDeserializer is single-result, so this stays 1:1 with the Kafka record; fanning one polled activity's events[] into N records is the poller's job (Phase C). toolEventId = uniqueQualifier + "-" + eventIndex. Failure predicate: eventName containing "_FAILURE" (case-insensitive) -> GWS_AUDIT_FAILURE before category mapping; unmapped eventType -> GWS_AUDIT_OTHER, never UNKNOWN. Ports the MS365 deserializer's 11 test invariants to the Google field map. Also backfills the .Microsoft365AuditEventDeserializer.md sidecar doc that was missing, per module convention. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t deserializer contract Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@openframe-stream-service-core/src/main/java/com/openframe/stream/deserializer/.Microsoft365AuditEventDeserializer.md`:
- Around line 50-53: Correct the Related Files link prefixes in
openframe-stream-service-core/src/main/java/com/openframe/stream/deserializer/.Microsoft365AuditEventDeserializer.md
lines 50-53: use openframe-stream-service-core/ for the stream-service links and
openframe-data-kafka/ for MessageType. Apply the same module prefixes to all
four links in
openframe-stream-service-core/src/main/java/com/openframe/stream/deserializer/.GoogleWorkspaceAuditEventDeserializer.md
lines 68-71.
In
`@openframe-stream-service-core/src/main/java/com/openframe/stream/deserializer/GoogleWorkspaceAuditEventDeserializer.java`:
- Around line 39-44: Update the parameter-value guidance in the deserializer
Javadoc to handle multiMessageValue explicitly: either include it in the
fallback expression after the existing value keys or document the required
structured handling so consumers do not drop multi-message parameters.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 124518aa-9e79-4137-b03c-0e2c55393b17
📒 Files selected for processing (10)
openframe-data-cassandra/src/main/java/com/openframe/data/cassandra/model/enums/UnifiedEventType.javaopenframe-data-kafka/src/main/java/com/openframe/data/model/enums/IntegratedToolType.javaopenframe-data-kafka/src/main/java/com/openframe/data/model/enums/MessageType.javaopenframe-stream-service-core/src/main/java/com/openframe/stream/deserializer/.GoogleWorkspaceAuditEventDeserializer.mdopenframe-stream-service-core/src/main/java/com/openframe/stream/deserializer/.Microsoft365AuditEventDeserializer.mdopenframe-stream-service-core/src/main/java/com/openframe/stream/deserializer/GoogleWorkspaceAuditEventDeserializer.javaopenframe-stream-service-core/src/main/java/com/openframe/stream/mapping/EventTypeMapper.javaopenframe-stream-service-core/src/main/java/com/openframe/stream/mapping/SourceEventTypes.javaopenframe-stream-service-core/src/test/java/com/openframe/stream/deserializer/GoogleWorkspaceAuditEventDeserializerTest.javaopenframe-stream-service-core/src/test/java/com/openframe/stream/mapping/EventTypeMapperTest.java
🚧 Files skipped from review as they are similar to previous changes (3)
- openframe-data-kafka/src/main/java/com/openframe/data/model/enums/IntegratedToolType.java
- openframe-data-kafka/src/main/java/com/openframe/data/model/enums/MessageType.java
- openframe-data-cassandra/src/main/java/com/openframe/data/cassandra/model/enums/UnifiedEventType.java
kirill-567
reviewed
Aug 11, 2026
| /** | ||
| * Microsoft 365 Entra directory audit event types (Graph directoryAudits {@code category} values). | ||
| */ | ||
| interface Microsoft365 { |
Contributor
There was a problem hiding this comment.
See no value at the interfaces.
Author
There was a problem hiding this comment.
It just practice that we had before. We already have Fleet, MeshCentral, Rmm interfaces there, so not to mess it up we should have separate interfaces here
Author
There was a problem hiding this comment.
if u want can create ticket for this refactoring
Contributor
There was a problem hiding this comment.
If it's convention of this service - that's fine
kirill-567
reviewed
Aug 11, 2026
| private Boolean isVisible; | ||
| private String tenantId; | ||
| private String organizationId; | ||
| private String organizationName; |
Contributor
There was a problem hiding this comment.
@Arsenii-Malov please verify if we need this update.
Author
There was a problem hiding this comment.
yes, because in microsoft and google logs we do not have any agent id (machineId) - it is just not linked. So we will not be able enrich those logs with organizationId, organizationName here. So i made that producer set it before message send, that why we need it here
Contributor
There was a problem hiding this comment.
Yes, It's ok
But I think we need to rename DeserializedDebeziumMessage -> DeserializedEventKafkaMessage
kirill-567
reviewed
Aug 11, 2026
| INTEGRATION_DELETED(Severity.INFO, "Integration deleted"), | ||
|
|
||
| // Microsoft 365 directory audit events | ||
| M365_USER_MANAGEMENT(Severity.INFO, "Microsoft 365 user management"), |
Contributor
There was a problem hiding this comment.
@Arsenii-Malov seams we get more and more events non related to the tools.
If we need to refactor events?
Contributor
There was a problem hiding this comment.
I think that all of them are events and it doesn't really matter whether they are tools or not
These .md contracts next to the deserializers are produced by the automated documentation job, so they should not be hand-carried in a feature PR. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Javadoc listed multiMessageValue among the Reports API parameter value keys but left it out of the "value ?? boolValue ?? intValue ?? multiValue" guidance, so a consumer following that chain could silently drop it. Spell out that multiMessageValue is intentionally excluded from the scalar fallback and needs recursive rendering of its nested parameter[] objects. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Additive oss-lib support for Microsoft 365 Entra directory audit events (Graph
auditLogs/directoryAudits) entering the existing logs pipeline (Cassandraunified_logs+ Pinot) via the genericmessage-typepath. First stage of the MS365 audit-logs feature; the poller/listener land in the saas repos after this releases.IntegratedToolType.MICROSOFT_365("microsoft-365")— enum name becomes the PinottoolType/ Logs UI filter valueMessageType.MICROSOFT_365_AUDIT_EVENT→CASSANDRA_EVENT_LOG+KAFKA_PINOT,COMMON_TYPEUnifiedEventTypeM365_*additions (role management + audit failures areWARNING, restINFO)EventTypeMappermappings for directoryAuditscategoryvalues (UserManagement, GroupManagement, ApplicationManagement, RoleManagement, Policy, DirectoryManagement)Microsoft365AuditEventDeserializer:toolEventId= Graph audit record id (idempotent upserts across the poller's cursor overlap window),eventTimestampfromactivityDateTime,result=failure→M365_AUDIT_FAILUREregardless of category, unmapped category →M365_AUDIT_OTHER, details =initiatedBy+targetResourcesPRE_ENRICHEDenrichment type +PreEnrichedDataEnrichmentService: audit events carry no agentId, andIntegratedToolDataEnrichmentServicepopulates org fields only via the agent→machine lookup — so pre-enriched payload fields (organizationId/organizationName/userId/tenantId, stamped by the per-org poller) pass through from the deserialized message instead.DeserializedDebeziumMessagegains the three fields additively.No behavior change for existing MessageTypes; oss-tenant deploys unaffected.
Test plan
EventTypeMapperTest— 6 category mappings, unmapped→UNKNOWN, tool-scopingMicrosoft365AuditEventDeserializerTest— field mapping, tenant/org passthrough, failure override, fallback event type, timestamp fallback, null-afterPreEnrichedDataEnrichmentServiceTest— passthrough + TenantIdProvider fallbackmvn testgreen🤖 Generated with Claude Code
Summary by CodeRabbit
New Features
Bug Fixes
Tests